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Abstract: Dual Coding Theory has quite specific predictions about how information in different media is 
stored, manipulated and recalled. Different combinations of media are expected to have significant effects 
upon the recall and retention of information. This obviously may have important consequences in the design 
of computer-based programmes. The paper describes an experimental approach which has been developed 
using the Statistical domain in which the presentation media have been varied (Text only. Text and Diagrams 
and Diagrams with Voice-over). The results are compared with Dual Coding theory prediaions and the 
effects of Student Learning Style explored. 



1. An Early Attempt at Using the Web to Communicate Statistics Knowledge 

Our interest in this work arose out of a desire to measure the effectiveness of Distance Learning compared with 
Face-to-face presentations. An opportunity arose in a second year HCI module which included two lectures on 
the use of the Null Hypothesis and Statistical Inference in HCI evaluation. We decided to divide the class into 
two groups, one of which would be taught face-to-face and the others over the Web. Presentation was mainly 
textual but some animations created using Visual Basic were also included. Students were first given a test to 
ascertain their knowledge of the material to be presented. There were some students with detailed knowledge, 
but most knew little about the domain. Throughout both presentations, questions were asked to enable students 
to check if they had understood the material. The Face-to-face class was taught the material on the last day of 
term, and the Distance Learners were given access to the material once they arrived home in the vacation. 

Access was protected by password and, becausethe student homes were widely dispersed it is unlikely that any 
of the “lecture” students could gain access to the material. On their return to the University in the next semester, 
the students were tested for knowledge of the material through a series of questions. We marked the test with a 
simple scoring system - one point for a correct answer. 



The results, as one might expect, were inconclusive. Overall, the performance of subjects on the post-test was 
similar for both groups. It was interesting that the Distance Learners did no worse overall that the Face-to-face 
Learners, though a number of them did report that they found working at home, alone, not very motivating. 

Scores varied widely (from 13 to 60). It was clear that some students in both groups had not taken the exercise 
seriously. It was also clear that previous knowledge had an effect (though students who professed previous 
knowledge did not always do well). Previous Mathematics expertise also had a positive effect on the score as 
might be expected. We had asked the Distance Learners to keep a log of their access to the system. Most did this 
and total access times reported varied from 1 hour to 6 hours. Some students also made comments on the way in 
which the material was presented. The material was heavily text -based and most comments suggested that more 
“interesting” media should have been used (whatever that meant). 
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Although this was only an exercise partly to illustrate to the students the problems involved in such 
experimentation, it did raise some major design issues for any future experiment: 

1 . By how far do different media affect the learning process? 

2. At what level should the learning material for the experiment be pitched and how extensive should it 
be? 

3. How can the students be m otivated so that they make a serious attempt at learning? 

4. How is the amount of material learned measured? Clearly correct answers to questions do give an 
indication, but how should the questions be phrased? 

5. How is previous knowledge allowed for? Are marks only given for improvements in knowledge 
transfer? Does someone who knows it all before get zero! 
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6. Should we be considering different learning styles? 



2. Effect of Different Media on Learning and Performance 

2.1 Some Process Control Results 

We have previously studied the effects of using different media on the performance of operators carrying out a 
laboratory task. The task chosen was that of Crossman’s Water Bath [Crossman & Cooke, 1974]. The task is 
closely related to the process domain [Sanderson et Al., 1989]. 

There was a marked difference in performance across media. For example, the performance results differed 
when using Text and Graphics. At low task complexity the performance was very similar, but as complexity 
increased the differences start to show. In the more complex tasks the results are significant (p<0.03) with 
Graphical interfaces giving the best performance. After each sequence of tests, the understanding of subjects 
was tested with a questionnaire. The comprehension results showed significant differences in the performance 
between the use of Verbal and Spatial Coding in the interface. 

Interestingly, spatial coding improved comprehension of the two least understood variables and this was a 
significant result (p<0.01). There is not space to report all the findings but the other variables showed similar 
trends (Alty, 1999). 

Thus the choice of medium used does seem to have an effect on comprehension and this could be important in 
learning situations. 

2.2 Dual Coding Theory 

Dual Coding Theory (Paivio, 1986) assumes an orthogonal relationship between symbolic systems and sensory 
motor systems. The theory suggests that there are clear distinctions between the internal representations of 
Symbolic and Sensory- motor events. The stored versions of visual, verbal, and haptic events retain the 
modalities of these events. For example, in the visual modality there are printed words and images. In the 
auditory modality there are spoken words and sound events. can be stored as an imagp of a lion, the word 
“lion”, or both but within distinct systems. Of major importance is the verbal/nonverbal distinction. The verbal 
and non-verbal processing systems are assumed to be ftmctionally independent though there are cross linkages 
between the two. If the Dual Coding theory is relevant then the recall of material will be affected by the way it is 
presented. 

3. Choice of Learning Domain and Subject Motivation 

Many experiments that have examined the effects of different media on learning have been constructed over 
relatively simple subject domains. The material being communicated to learners is often limited in scope and 
usually not complex in nature. There are, of course, good reasons for this. A complex domain necessarily 
requires a specialised user base, and an extensive set of learning material will impose serious time requirements 
on the subjects taking part. Yet it is important to really challenge subjects both with domain complexity and the 
extent of the material in order to obtain results which will scale-up for real situations. 

One possible solution to this problem is to choose a domain that is acknowledged to be inherently difficult, is 
difficult to teach, and yet for many people, constitutes a most desirable skill to attain. Within a University 
context (the population from we from which we are most likely to draw subjects) there was an obvious 
candidate - Statistics. 

Most Masters and PhD. students require statistical knowledge for analysing their experiments, and yet it is a 
subject most feared by many of them. Thus the Statisitical Domain fulfils our requirements of being complex, of 
being not easy to teach, and yet would be regarded by subjects as a required and desirable skill. There should 
therefore be a strong motivation to take such a course and to take it seriously. Furthermore, at Loughborough 
University, we have a Masters course on Multimedia Interface Design, and an important aspect of the course is 
the design and evaluation of HCI experiments using Statistics. The material is typically taught in four one-hour 
lectures on the course and covers basic information about the Null Hypothesis, the Binomial Distribution, the 
Sign Test, the Wilcoxon Ranking Non-parametric Test and Normal distributions and their use. We therefore 
decided to develop Web-based material to teach the subject matter on the course and to additionally make the 
exercise an example of HCI experimentation and its evaluation. 
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We constructed four separate computer-based modules in FLASH to teach this basic statistical knowledge: 

1. The Null Hypothesis and the relevance of Statistics 

2. The Binomial Distribution 

3. Non -parametric distributions - i.e. Ranking (Wilcoxon) 

4. The Normal Distribution and the Central Limit Theorem 

The presentations were constructed using three different media combinations of voice, text and 
diagrams/pictures. The material and timing was identical in each presentation. In other words, the text and the 
voice-over content were identical. Each of these media could be disabled so that a number of different 
presentation formats was possible. We actually used: 

1 . Text only 

2. Text + Diagrams 

3. Voice + Diagrams 

These formats were preserved over a complete module presentation session. 

4. Proposed Experimental Approach 

Each developed presentation had an elapse time of about 10-15 minutes and each was presented in a serial non - 
interactive fashion. We planned to place subjects in a room and present the material on a screen using a 
computer projector. At the commencement of the first session all subjects were to be given a short questionnaire 
to determine their previous knowledge of the subject and would be told to write nothing down during the 
experiment, but that at the conclusion of the experiment they would be required to answer a series of questions 
on the material. If students were subjects, they would be assured that the results did not form any part of the 
evaluation process for their module 

The subjects would be tested for immediate recall of the material presented at the conclusbn of the presentation. 
The post experimental test would consist of a series of questions about the presented material. In order to allow 
for the effect of previous knowledge, subjects would be able to indicate alongside their answer whether they 
already knew the answer before the session, whether the session had reminded them of the answer, or whether 
the material in the question was completely new to them. 

5. The Construction of the Material 

Because all the three different media presentations had to be based upon identical material, the construction of 
the different sessions was an interesting exercise in itself. When constructing the Voice-over + Diagrams 
presentation we had to be mindful of the fact that the Voice-over (as text) had also to work when presented as a 
Text -only session. This exercise in itself provides useful insights into the nature of multimedia presentations, 
how media are implicitly allocated for communicating particular types of material and the limitations some 
media have for communicating particular types of material. For example, communicating the idea that the 
results of an experiment might be caused by a set of independent random events is quite straightforward in text, 
but is not easy to do in a diagram. On the other hand some material (describing the shape of a normal 
distribution) is easy with a diagram but laborious using text. The Null Hypothesis concept is relatively 
straightforward to communicate using text but more difficult using a diagram. All the material was constructed 
in FLASH and each module was divided into a number of FLASH scenes. The scenes were transparent to the 
subjects, but we used this approach so that later the material could be adapted so that it could be presented in a 
more parallel, interactive manner at a future date. 

6. An Initial Experiment 

The first module was actually constructed in Australia whilst the author was a Visiting Research Fellow at 
Melbourne University Department of Information Systems. Towards the end of the Fellowship the author gave a 
commercial course on Multimedia Interface Design to a class of about 50 commercial, industrial and academic 
participants, which lasted two days. It was decided that this would be a good opportunity to run a pilot study of 
the first module during the course to give the participants an insight into evaluation techniques and to 
demonstrate the effects of different media on learning. From the point of view of the experiment, it would 



provide an exercise in creating the material, a test of the suitability of the material and provide experience in 
setting the post-experiment test. The participants (most of whom in the pre-test claimed not to have significant 
knowledge in the Statistics area) were divided into three groups and presented with the three different 
presentation styles - Text-only, Text + Diagrams and Voice-over + Diagrams, 

All presentations were given in separate room areas using a computer projector. At the end of the test all 
subjects answered the same test to check recall of material. No learning style classification of users was 
attempted in this pilot run. The test administered at the conclusion of the experiment consisted of a series of 
questions, some of which were multiple- choice questions. 




Figure 1: Raw results from the Australian Exponment on Recall. 



The results are shown in (Fig 1). The three vertical columns refer to whether the subject had met the domain or 
concepts before (A: Knew it, B.’ Previously learned and forgotten, but reminded, C:Did not know it). One can 
clearly see the effect of previous knowledge. The marks decrease as subject knowledge decreases. This 
experiment was only carried out as a pilot, to guide future experimental design (for example, we did not expect 
to carry out any statistical analysis on the result) the results seemed initially to contradict Dual Coding Theory 
predictions. For example, the Text Only group (A) had a tendency to do better than the Voice-over-Diagrams 
group (B), whereas we might have expected the results to be in the opposite direction. However, a replay of the 
experimental material discovered an error. We had met with some trouble in starting the three presentations and, 
in the confusion, had actually presented Voice -Over+Text (not Voice Over+Diagrams) to Group (B). Thus in 
retrospect it was not surprising that the Group (B) performed badly. Previous experiments have shown that the 
redundancy of Text and Voice-over can worsen performance. Other interesting features were noted. Subjects 
who professed to know the material often still made recall errors. Finally we had great difficulty in analyzing the 
questionnaires in giving marks for recall and realized that the questions needed to be more carefully designed 
and related more carefully to the material. 

7. The First Student Class Experiment 

We reorganized the material so that each scene had clear learning objectives and the resulting questions were 
derived from these objectives. Some questions were carefully chosen such that even previous knowledge in 
itself would not enable a student to answer it. We also decided to test the students Learning Style using the 
Felder and Soloman Test (Felder 1993). The test identifies preferences on the axes Active/Reflective, 
Sensing/Intuitive, Visual/Verbal and Sequential/Global. 

We asked students on a first year HCI course to take the learning style test. The spread of learning styles 
indicated that there were similar sized groups of Sensing Learners and Intuitive Learners in the class, and the 
rest of the students had Learning Styles balanced between the two. The class was then divided into three groups 
each group having an equal number of Sensing Learners, Intuitive Learners and Balanced subjects. A further 
regrouping improved the gender balance without disturbing the Learning Style balance. 

The first module (Null Hypothesis) was then presented to the three groups of students -Text -only. Text and 
Diagrams, and Diagrams with Voice over. The results of this exercise are reported elsewhere in this conference 
(Beacham, Elliott, Alty and AlSharrah, 2002). The media combination used was significant with respect to 
recalled material and it was shown that Learning style played an important part in the accuracy of recall. 
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8. The Extended Student Class Experiment 



As a result of the success of the first experimentwe completed all four modules and tested them on the students 
attending a postgraduate Multimedia Interface Design course (59 students, though all did not take part in all 
sessions). The students answered a questionnaire to determine their learning style. The test administered was 
again that of Felder and Soloman. They were then again divided into three groups (A, B and C) according to 
their learning style SENSING / INTUITIVE / BALANCED. 

As far as possible, each group was balanced for gender and learning styles. Groups were given the four modules 
(in the different presentation formats) as detailed below in (Tab. 1). 





Text 


Text + Diagrams 


Voice + Diagrams 


Null Hypothesis 


Group A 


Group B 


Group C 


Binomial Distribution 


Group B 


Group C 


Group D 


Ranking 


Group C 


Group A 


Group B 


Normal Distribution 


Group A 


Group B 


Group C 



Table 1: Groups and Presentation Formats 



Since learning about Statistics was part of the course as well as the experiment, we did not want to disadvantage 
any students who might do less well on a particular presentation technique. We also did not want succeeding 
modules to be affected by any lack of knowledge gained in previous modules (though generally this was not a 
problem because most of the material was quite different). All students were therefore given a normal lecture 
on the material of a module after the presentation. They were able to interact and ask questions to ensure they 
fully understood the material. Any students who did not complete all sessions were removed from the sampb. 
This left 37 students. 




Expt 1 Expt 2 Expt 3 Expt 4 Overall 
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■ text only 



Figure 2: Preliminary Experimental Results 



The results for the different media can be seen in (Fig. 2). In all cases the predictions of Dual Coding theory are 
borne out 



Our sample included some dyslexic learners. Although the sample was too small to be significant (6 subjects), 
the results did suggest that such learners might react differently see (Fig. 3), and this looks like an interesting 
area for more experimentation. The dyslexic subjects do not seem to follow the predictions of Dual Coding 
Theory (or at least there seems to be more complex interactions taking place). The results also indicated that the 
material based upon real world objects were recalled more accurately than more abstract visual material. 

Certain learning styles were influenced more (or less) by different media (Fig. 4). For example, Intuitive 
Learners tended to out -perform Sensing Learners in all presentations, although in certain experiments. Sensing 
Learners using Voice/Diagrams outperformed Intuitive Learners using Text -only. 



Mean learning scores for dyslexics and non-dyslexics 




□ sound and diagram!; 
Qtext and diagrams 
Btext only 

DL Dyslexic 
NDL Non-Dyslexic 



Figure 3: The Preliminary results for Dyslexia 



Mean learning scores for learning styles 
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Expt 2 


Expt 3 
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f~~j sound and diagrams 
Q text and diagrams 
H text only 
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I Intuitive 



Figure 4: Performance for Different Learning Styles 
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